{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": []
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "# How to Perform RAG with the New Open-Source Library 'aisuite' for Financial Analysis?\n",
        "\n",
        "In this notebook, I explore how to streamline financial analysis using a combination of the new open-source library \"aisuite\", LlamaParse, a vector-based approach, and powerful LLMs like GPT-4o and Claude Sonnet 3.5. Here's a breakdown:\n",
        "\n",
        "\n",
        "\n",
        "\n",
        "✅ The Workflow:\n",
        "\n",
        "\n",
        "\n",
        "1- Parsing Financial PDFs with LlamaParse:\n",
        "\n",
        " I used LlamaParse to convert Amazon’s 2023 10-K report into a structured format for analysis.\n",
        "\n",
        "\n",
        "\n",
        "2- Chunking for Retrieval:\n",
        "\n",
        " Using a SentenceSplitter, I divided the report into chunks optimized for embedding and retrieval.\n",
        "\n",
        "\n",
        "\n",
        "3- Building a Vector Index:\n",
        "\n",
        " I created a vector store using the local embedding model BAAI/bge-small-en-v1.5, ensuring full control.\n",
        "\n",
        "\n",
        "\n",
        "4- Querying Financial Data with RAG:\n",
        "\n",
        " Questions like \"What was the total lease cost for 2023?\" and \"What was the net income in 2023?\" were answered by combining context retrieval and language model synthesis.\n",
        "\n",
        "\n",
        "\n",
        "5- Comparing LLM Outputs:\n",
        "\n",
        " Using aisuite, I ran the queries through both GPT-4o and Claude for robust, multi-perspective answers.\n",
        "\n",
        "Claude/GPT-4o: Total lease cost: $18.918 billion.\n",
        "\n",
        "Claude/GPT-4o: Net income: $30.437 billion.\n",
        "\n",
        "\n",
        "✅ Why It Matters:\n",
        "\n",
        "\n",
        "With RAG workflows powered by \"aisuite\", we can automate the tedious process of sifting through dense financial reports, and comparing results from different LLMs.\n",
        "\n",
        "This approach not only accelerates analysis but also ensures accuracy, empowering analysts to focus on decision-making.\n"
      ],
      "metadata": {
        "id": "OFDiWDTMgzV0"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "[Hanane D](https://www.linkedin.com/in/hanane-d-algo-trader)"
      ],
      "metadata": {
        "id": "xPgmAFfxlBGQ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install aisuite[all] -q"
      ],
      "metadata": {
        "id": "NBaOoqLICCd5"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from google.colab import userdata\n",
        "OPENAI_API_KEY = userdata.get('OPENAI_API_KEY')\n",
        "CLAUDE_API_KEY = userdata.get('CLAUDE_API_KEY')\n",
        "LLAMAPARSE_API_KEY = userdata.get('LLAMACLOUD_API_KEY')"
      ],
      "metadata": {
        "id": "lDSnHAGmCE5I"
      },
      "execution_count": 6,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "# from getpass import getpass\n",
        "os.environ['OPENAI_API_KEY'] = OPENAI_API_KEY\n",
        "os.environ['ANTHROPIC_API_KEY'] = CLAUDE_API_KEY"
      ],
      "metadata": {
        "id": "7yzSwjn3CHF4"
      },
      "execution_count": 4,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "!pip install llama-index llama-index-core llama-parse llama_index.embeddings.huggingface -q\n",
        "# !pip install llama-index-llms-anthropic -q"
      ],
      "metadata": {
        "id": "7WmB-9lICX3b"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Load 10k financial report of Amazon"
      ],
      "metadata": {
        "id": "R7-FeICIaK69"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget \"https://d18rn0p25nwr6d.cloudfront.net/CIK-0001018724/c7c14359-36fa-40c3-b3ca-5bf7f3fa0b96.pdf\" -O amzn_2023_10k.pdf"
      ],
      "metadata": {
        "id": "X9iC1kH1aQLW"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# LLamaParse"
      ],
      "metadata": {
        "id": "PaowEp8naR2O"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Parse Amazon 10k report using LlamaParse"
      ],
      "metadata": {
        "id": "VhaFEwTGaeC4"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_parse import LlamaParse\n",
        "import nest_asyncio;\n",
        "nest_asyncio.apply()\n",
        "\n",
        "pdf_name = \"amzn_2023_10k.pdf\"\n",
        "parser = LlamaParse(api_key=LLAMAPARSE_API_KEY, result_type=\"markdown\", gpt4o_mode = True)\n",
        "documents = parser.load_data(pdf_name)"
      ],
      "metadata": {
        "id": "0WWo_xgnCTSr"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Embedding + VectorStore"
      ],
      "metadata": {
        "id": "JOp-S1AZajqY"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Using  BAAI/bge-small-en-v1.5 in HuggingFace for embedding. You need HF_TOKEN."
      ],
      "metadata": {
        "id": "SDq5rioRaoRJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index.core.node_parser import SentenceSplitter\n",
        "\n",
        "######## SentenceSplitter ########\n",
        "splitter = SentenceSplitter(chunk_size=1024)\n",
        "nodes = splitter.get_nodes_from_documents(documents)\n",
        "\n",
        "######## Vector Index ########\n",
        "from llama_index.core import VectorStoreIndex\n",
        "\n",
        "embed_model = \"local:BAAI/bge-small-en-v1.5\" #https://huggingface.co/collections/BAAI/bge-66797a74476eb1f085c7446d\n",
        "vector_index = VectorStoreIndex(nodes, embed_model = embed_model)"
      ],
      "metadata": {
        "id": "xD0IGRcoCcws"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Retriever"
      ],
      "metadata": {
        "id": "dDE_AjEJa5nz"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Retrieve the most relevant context for a user query based on the embedded information in vectorstore:"
      ],
      "metadata": {
        "id": "HHk8fPNLbcCM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index.core.indices.vector_store.retrievers import VectorIndexRetriever\n",
        "from llama_index.core.query_engine.retriever_query_engine import RetrieverQueryEngine\n",
        "from llama_index.core import get_response_synthesizer\n",
        "\n",
        "retriever = VectorIndexRetriever(\n",
        "    index=vector_index,\n",
        "    similarity_top_k=3,\n",
        ")\n",
        "# build query engine\n",
        "query_engine = RetrieverQueryEngine(\n",
        "    retriever=retriever\n",
        ")\n",
        "\n",
        "response = query_engine.query(\"what was the net income in 2023?\")"
      ],
      "metadata": {
        "id": "nRG8hDuGRYjr"
      },
      "execution_count": 19,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [],
      "metadata": {
        "id": "D3jnwp0TRVyb"
      },
      "execution_count": 40,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Final context"
      ],
      "metadata": {
        "id": "4PhJvO7hb_zB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Display the context of the 3 nodes extracted:"
      ],
      "metadata": {
        "id": "xHUcZtuVbl1O"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "i=0\n",
        "context=\"\"\n",
        "for node in response.source_nodes:\n",
        "  print(f\"node {i}\")\n",
        "  # print(node.text)\n",
        "  context += node.text\n",
        "  context += \"\\n\" + \"=\"*50 + \"\\n\"  # Separator for each page\n",
        "  i+=1"
      ],
      "metadata": {
        "id": "2eD-JXpSTPpH"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "context"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 122
        },
        "id": "BsnM9mO6Rpyu",
        "outputId": "e04a358f-9a23-4c44-acc5-12454c6eb260"
      },
      "execution_count": 46,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "'# The components of the provision (benefit) for income taxes, net are as follows (in millions):\\n\\n| Year Ended December 31, | 2021  | 2022   | 2023   |\\n|-------------------------|-------|--------|--------|\\n| **U.S. Federal:**       |       |        |        |\\n| Current                 | $2,129| $2,175 | $8,652 |\\n| Deferred                | 155   | (6,686)| (5,505)|\\n| Total                   | 2,284 | (4,511)| 3,147  |\\n| **U.S. State:**         |       |        |        |\\n| Current                 | 763   | 1,074  | 2,158  |\\n| Deferred                | (178) | (1,302)| (498)  |\\n| Total                   | 585   | (228)  | 1,660  |\\n| **International:**      |       |        |        |\\n| Current                 | 2,209 | 1,682  | 2,186  |\\n| Deferred                | (287) | (160)  | 127    |\\n| Total                   | 1,922 | 1,522  | 2,313  |\\n| **Provision (benefit) for income taxes, net** | $4,791| $(3,217)| $7,120 |\\n\\n# U.S. and international components of income (loss) before income taxes are as follows (in millions):\\n\\n| Year Ended December 31, | 2021   | 2022    | 2023   |\\n|-------------------------|--------|---------|--------|\\n| **U.S.**                | $35,879| $(8,225)| $32,328|\\n| **International**       | 2,272  | 2,289   | 5,229  |\\n| **Income (loss) before income taxes** | $38,151| $(5,936)| $37,557|\\n\\n# The items accounting for differences between income taxes computed at the federal statutory rate and the provision (benefit) recorded for income taxes are as follows (in millions):\\n\\n| Year Ended December 31, | 2021   | 2022    | 2023   |\\n|-------------------------|--------|---------|--------|\\n| **Income taxes computed at the federal statutory rate** | $8,012 | $(1,246)| $7,887 |\\n| **Effect of:**          |        |         |        |\\n| Tax impact of foreign earnings and losses | (1,349)| (370)  | 594    |\\n| State taxes, net of federal benefits | 465    | (173)  | 1,307  |\\n| Tax credits             | (1,136)| (1,006) | (2,362)|\\n| Stock-based compensation (1) | (1,094)| 612    | 1,047  |\\n| Foreign income deduction (2) | (301)  | (1,258)| (1,429)|\\n| Other, net              | 194    | 224     | 76     |\\n| **Total**               | $4,791 | $(3,217)| $7,120 |\\n\\n(1) Includes non-deductible stock-based compensation and excess tax benefits or shortfalls from stock-based compensation. Our tax provision includes $1.9 billion of excess tax benefits from stock-based compensation for 2021, and $33 million and $519 million of tax shortfalls from stock-based compensation for 2022 and 2023.\\n\\n(2) U.S. companies are eligible for a deduction that lowers the effective tax rate on certain foreign income. This regime is referred to as the Foreign-Derived Intangible Income deduction and is dependent on the amount of our U.S. taxable income.\\n\\nWe generated an income tax benefit in 2022 as compared to a provision for income taxes in 2021 primarily due to a decrease in pretax income and an increase in the foreign income deduction. This was partially offset by a reduction in excess tax benefits from stock-based compensation and a decrease in the tax impact of foreign earnings and losses driven by a decline in the favorable effects of corporate restructuring transactions. The foreign income deduction benefit recognized in 2022 reflects a change in our application of tax regulations related to the computation of qualifying foreign income and includes a tax benefit of approximately $655 million related to years prior to 2022.\\n==================================================\\n# Interest Income and Expense\\n\\nOur interest income was $989 million and $2.9 billion during 2022 and 2023, primarily due to an increase in prevailing rates. We generally invest our excess cash in AAA-rated money market funds and investment grade short- to intermediate-term marketable debt securities. Our interest income corresponds with the average balance of invested funds based on the prevailing rates, which vary depending on the geographies and currencies in which they are invested.\\n\\nInterest expense was $2.4 billion and $3.2 billion in 2022 and 2023 and was primarily related to debt and finance leases. See Item 8 of Part II, “Financial Statements and Supplementary Data — Note 4 — Leases and Note 6 — Debt” for additional information.\\n\\nOur long-term lease liabilities were $73.0 billion and $77.3 billion as of December 31, 2022 and 2023. Our long-term debt was $67.1 billion and $58.3 billion as of December 31, 2022 and 2023. See Item 8 of Part II, “Financial Statements and Supplementary Data — Note 4 — Leases and Note 6 — Debt” for additional information.\\n\\n# Other Income (Expense), Net\\n\\nOther income (expense), net was $(1.6) billion and $938 million during 2022 and 2023. The primary components of other income (expense), net are related to equity securities valuations and adjustments, equity warrant valuations, and foreign currency. Included in other income (expense), net in 2022 and 2023 is a marketable equity securities valuation gain (loss) of $(1.2) billion and $797 million from our equity investment in Rivian.\\n\\n# Income Taxes\\n\\nOur effective tax rate is subject to significant variation due to several factors, including variability in our pre-tax and taxable income and loss and the mix of jurisdictions where we operate, intercompany transactions, the applicability of special tax regimes, changes in how we do business, acquisitions, investments, developments in tax controversies, changes in our stock price, changes in our deferred tax assets and liabilities and their valuation, foreign currency gains (losses), changes in tax reserves, case law, and administrative practices, principles, and interpretations related to tax, including changes to the basis of taxation, tax rates, income, expenses, and accounting rules in various jurisdictions, and relative changes of expenses or losses for which tax benefits are not recognized. Our effective tax rate is more or less volatile based on the amount of pre-tax income or loss. For example, the impact of discrete items to the effective tax rate is greater when our pre-tax income is lower. In addition, we record valuation allowances against deferred tax assets when there is uncertainty about our ability to generate future income in relevant jurisdictions.\\n\\nWe recorded a provision (benefit) for income taxes of $(3.2) billion and $7.1 billion in 2022 and 2023. See Item 8 of Part II, “Financial Statements and Supplementary Data — Note 9 — Income Taxes” for additional information.\\n\\n# Non-GAAP Financial Measures\\n\\nRegulation G, Conditions for Use of Non-GAAP Financial Measures, and other SEC regulations define and prescribe the conditions for use of certain non-GAAP financial information. Our measures of free cash flows and the effect of foreign exchange rates on our consolidated statements of operations meet the definition of non-GAAP financial measures.\\n\\nWe provide multiple measures of free cash flows because we believe these measures provide additional perspective on the impact of acquiring property and equipment with cash and through finance leases and financing obligations.\\n\\n## Free Cash Flow\\n\\nFree cash flow is cash flow from operations reduced by “Purchases of property and equipment, net of proceeds from sales and incentives.” The following is a reconciliation of free cash flow to the most comparable GAAP cash flow measure, “Net cash provided by (used in) operating activities,” for 2022 and 2023 (in millions):\\n\\n|                                      | Year Ended December 31, |\\n|--------------------------------------|-------------------------|\\n|                                      | 2022       | 2023       |\\n| Net cash provided by (used in) operating activities | $46,752    | $84,946    |\\n| Purchases of property and equipment, net of proceeds from sales and incentives | $(58,321)  | $(48,133)  |\\n| Free cash flow                       | $(11,569)  | $36,813    |\\n| Net cash provided by (used in) investing activities | $(37,601)  | $(49,833)  |\\n| Net cash provided by (used in) financing activities | $9,718     | $(15,879)  |\\n==================================================\\n# Guidance\\n\\nWe provided guidance on February 1, 2024, in our earnings release furnished on Form 8-K as set forth below. These forward-looking statements reflect Amazon.com’s expectations as of February 1, 2024, and are subject to substantial uncertainty. Our results are inherently unpredictable and may be materially affected by many factors, such as fluctuations in foreign exchange rates, changes in global economic and geopolitical conditions and customer demand and spending (including the impact of recessionary fears), inflation, interest rates, regional labor market constraints, world events, the rate of growth of the internet, online commerce, cloud services, and new and emerging technologies, as well as those outlined in Item 1A of Part 1, “Risk Factors.”\\n\\n## First Quarter 2024 Guidance\\n\\n- Net sales are expected to be between $138.0 billion and $143.5 billion, or to grow between 8% and 13% compared with first quarter 2023. This guidance anticipates a favorable impact of approximately 40 basis points from foreign exchange rates.\\n\\n- Operating income is expected to be between $8.0 billion and $12.0 billion, compared with $4.8 billion in first quarter 2023. This guidance includes approximately $0.9 billion lower depreciation expense due to an increase in the estimated useful life of our servers beginning on January 1, 2024.\\n\\n- This guidance assumes, among other things, that no additional business acquisitions, restructurings, or legal settlements are concluded.\\n==================================================\\n'"
            ],
            "application/vnd.google.colaboratory.intrinsic+json": {
              "type": "string"
            }
          },
          "metadata": {},
          "execution_count": 46
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# aisuite"
      ],
      "metadata": {
        "id": "Q2nqPulZbsZH"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Define the list of the llms you want to use/compare:"
      ],
      "metadata": {
        "id": "7ZT_-i2QbutH"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import aisuite as ai\n",
        "import time\n",
        "\n",
        "client = ai.Client()\n",
        "\n",
        "llms = [\n",
        "        \"anthropic:claude-3-5-sonnet-20241022\",\n",
        "        \"openai:gpt-4o\",\n",
        "       ]"
      ],
      "metadata": {
        "id": "DfO4uIccCJcK"
      },
      "execution_count": 47,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Chat"
      ],
      "metadata": {
        "id": "TrTxYT_Ub7cR"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Include the context and the query in the message to be sent to the LLM:"
      ],
      "metadata": {
        "id": "WF8-TOHkbzVg"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "messages = [\n",
        "    {\"role\": \"system\", \"content\": \"You are an expert in financial analysis.\"},\n",
        "]\n",
        "\n",
        "query = \"What was the net income on 2023?\"\n",
        "user_content = f\"\"\"Based on the following context, answer the question: \\n\\nContext:{context}\\n\\nQuestion:{query}\\n\\nAnswer:\"\"\"\n",
        "messages.append( {\"role\": \"user\", \"content\": user_content} )\n",
        "messages"
      ],
      "metadata": {
        "collapsed": true,
        "id": "QJDTF_9LC2He"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Ask the LLMs to answer"
      ],
      "metadata": {
        "id": "dRi7uRDBcFWK"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for llm_aisuite in llms[:]:\n",
        "  print(f\"LLM: {llm_aisuite}\")\n",
        "  response = client.chat.completions.create(model=llm_aisuite, messages=messages)\n",
        "  print(response.choices[0].message.content)\n",
        "  print(\"\\n\\n\")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "765KqmK8DEL_",
        "outputId": "8ee3bec2-023c-48e2-b947-6befa0f91dd0"
      },
      "execution_count": 53,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "LLM: anthropic:claude-3-5-sonnet-20241022\n",
            "Based on the provided context, I can calculate the net income for 2023:\n",
            "\n",
            "Income before income taxes: $37,557 million\n",
            "Minus Provision for income taxes: $7,120 million\n",
            "= Net income of $30,437 million (or approximately $30.4 billion) for 2023.\n",
            "\n",
            "This calculation is derived by taking the income before income taxes and subtracting the provision for income taxes for that year.\n",
            "\n",
            "\n",
            "\n",
            "LLM: openai:gpt-4o\n",
            "To calculate the net income for 2023, we use the formula:\n",
            "\n",
            "\\[ \\text{Net Income} = \\text{Income Before Income Taxes} - \\text{Provision for Income Taxes, Net} \\]\n",
            "\n",
            "From the provided data:\n",
            "- Income (loss) before income taxes for 2023: $37,557 million\n",
            "- Provision for income taxes, net for 2023: $7,120 million\n",
            "\n",
            "Substituting these values into the formula gives:\n",
            "\n",
            "\\[ \\text{Net Income} = 37,557 - 7,120 = 30,437 \\]\n",
            "\n",
            "Therefore, the net income for 2023 was $30,437 million.\n",
            "\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# All Together"
      ],
      "metadata": {
        "id": "Z36ov0oZXQYp"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget \"https://d18rn0p25nwr6d.cloudfront.net/CIK-0001018724/c7c14359-36fa-40c3-b3ca-5bf7f3fa0b96.pdf\" -O amzn_2023_10k.pdf\n",
        "\n",
        "from llama_parse import LlamaParse\n",
        "import nest_asyncio;\n",
        "nest_asyncio.apply()\n",
        "\n",
        "######## Parse the pdf with LlamaParse ########\n",
        "pdf_name = \"amzn_2023_10k.pdf\"\n",
        "parser = LlamaParse(api_key=LLAMAPARSE_API_KEY, result_type=\"markdown\", gpt4o_mode = True)\n",
        "documents = parser.load_data(pdf_name)\n",
        "\n",
        "from llama_index.core.node_parser import SentenceSplitter\n",
        "\n",
        "######## SentenceSplitter ########\n",
        "splitter = SentenceSplitter(chunk_size=1024)\n",
        "nodes = splitter.get_nodes_from_documents(documents)\n",
        "\n",
        "######## Vector Index ########\n",
        "from llama_index.core import VectorStoreIndex\n",
        "\n",
        "embed_model = \"local:BAAI/bge-small-en-v1.5\" #https://huggingface.co/collections/BAAI/bge-66797a74476eb1f085c7446d\n",
        "vector_index = VectorStoreIndex(nodes, embed_model = embed_model)\n",
        "\n",
        "######## Retriver ########\n",
        "from llama_index.core.indices.vector_store.retrievers import VectorIndexRetriever\n",
        "from llama_index.core.query_engine.retriever_query_engine import RetrieverQueryEngine\n",
        "from llama_index.core import get_response_synthesizer\n",
        "\n",
        "retriever = VectorIndexRetriever(\n",
        "    index=vector_index,\n",
        "    similarity_top_k=3,\n",
        ")\n",
        "# build query engine\n",
        "query_engine = RetrieverQueryEngine(\n",
        "    retriever=retriever\n",
        ")\n",
        "\n",
        "# response = query_engine.query(\"what was the net income in 2023?\")"
      ],
      "metadata": {
        "id": "1S_wZHQYcYqk"
      },
      "execution_count": 76,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def get_context(query_engine,user_query):\n",
        "  # query_engine = RetrieverQueryEngine(\n",
        "  #     retriever=retriever, response_synthesizer=get_response_synthesizer()\n",
        "  # )\n",
        "  response = query_engine.query(user_query)\n",
        "\n",
        "  context=\"\"\n",
        "  for node in response.source_nodes:\n",
        "    context += node.text\n",
        "    context += \"\\n\" + \"=\"*100 + \"\\n\"\n",
        "\n",
        "  return context"
      ],
      "metadata": {
        "id": "FVcbV_pbXOJj"
      },
      "execution_count": 77,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "def create_messages(query_engine, user_query, system_content = \"You are an expert in financial analysis.\"):\n",
        "\n",
        "  context = get_context(query_engine,user_query)\n",
        "\n",
        "  messages = [\n",
        "      {\"role\": \"system\", \"content\": system_content},\n",
        "  ]\n",
        "\n",
        "  user_content = f\"\"\"Based on the following context, answer the question: \\n\\nContext:{context}\\n\\nQuestion:{user_query}\\n\\nAnswer:\"\"\"\n",
        "  messages.append( {\"role\": \"user\", \"content\": user_content} )\n",
        "  return messages\n",
        "\n",
        "def call_llm_aisuite(llms, user_query):\n",
        "\n",
        "  messages = create_messages(query_engine, user_query)\n",
        "\n",
        "  for llm_aisuite in llms:\n",
        "    print(f\"LLM: {llm_aisuite}\")\n",
        "    response = client.chat.completions.create(model=llm_aisuite, messages=messages)\n",
        "    print(response.choices[0].message.content)\n",
        "    print(\"\\n\\n\")"
      ],
      "metadata": {
        "id": "GNbFJtPWXmRE"
      },
      "execution_count": 80,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "user_query= \"What was the total lease cost for the year ended December 31, 2023?\"\n",
        "\n",
        "client = ai.Client()\n",
        "\n",
        "llms = [\n",
        "        \"anthropic:claude-3-5-sonnet-20241022\",\n",
        "        \"openai:gpt-4o\",\n",
        "       ]\n",
        "\n",
        "call_llm_aisuite(llms, user_query)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "XW2jKqHKZhZy",
        "outputId": "1d7002f6-aa19-444d-f33c-68fe61cbf57f"
      },
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "LLM: anthropic:claude-3-5-sonnet-20241022\n",
            "According to the lease cost summary table in the context, the total lease cost for the year ended December 31, 2023 was $18,918 million. This total comprises:\n",
            "- Operating lease cost: $10,550 million\n",
            "- Finance lease cost: $6,203 million (made up of $5,899 million in amortization of lease assets and $304 million in interest on lease liabilities)\n",
            "- Variable lease cost: $2,165 million\n",
            "\n",
            "\n",
            "\n",
            "LLM: openai:gpt-4o\n",
            "The total lease cost for the year ended December 31, 2023, was $18,918 million.\n",
            "\n",
            "\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "user_query= \"What was the net income on 2023?\"\n",
        "\n",
        "client = ai.Client()\n",
        "\n",
        "llms = [\n",
        "        \"anthropic:claude-3-5-sonnet-20241022\",\n",
        "        \"openai:gpt-4o\",\n",
        "       ]\n",
        "\n",
        "call_llm_aisuite(llms, user_query)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lpydSVumdK38",
        "outputId": "055f05a9-37f2-4fb1-a29f-00845d43f5a1"
      },
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "LLM: anthropic:claude-3-5-sonnet-20241022\n",
            "To calculate the net income for 2023, I'll subtract the provision for income taxes from the income before income taxes:\n",
            "\n",
            "Income before income taxes (2023): $37,557 million\n",
            "Less: Provision for income taxes (2023): $7,120 million\n",
            "Net Income (2023): $37,557 - $7,120 = $30,437 million (or $30.437 billion)\n",
            "\n",
            "The company showed strong profitability in 2023, with a significant improvement from 2022 when they had a loss before income taxes of $5,936 million.\n",
            "\n",
            "\n",
            "\n",
            "LLM: openai:gpt-4o\n",
            "To determine the net income for the year 2023, we need to use the formula:\n",
            "\n",
            "\\[ \\text{Net Income} = \\text{Income before Income Taxes} - \\text{Provision for Income Taxes, Net} \\]\n",
            "\n",
            "From the provided data for the year ending December 31, 2023:\n",
            "\n",
            "- **Income before Income Taxes**: $37,557 million\n",
            "- **Provision for Income Taxes, Net**: $7,120 million\n",
            "\n",
            "Plugging these values into our formula, we get:\n",
            "\n",
            "\\[ \\text{Net Income} = 37,557 - 7,120 = 30,437 \\]\n",
            "\n",
            "Therefore, the net income for the year 2023 was $30,437 million.\n",
            "\n",
            "\n",
            "\n"
          ]
        }
      ]
    }
  ]
}